Speeding Up Relational Data Mining by Learning to

he motivation behind multi-relational data mining is wledge discovery in relational databases containing tiple related tables. One difficulty relational data ing faces is managing intractably large hypothesis ces. We attempt to overcome this difficulty by first pling the hypothesis space. We generate a small set of otheses, uniformly sampled from the space of didate hypotheses, and evaluate this set on actual . These hypotheses and their corresponding luation scores serve as training data in learning an roximate hypothesis evaluator. We use this roximate evaluation to quickly rate potential otheses without needing to score them on actual data. test our approximate clause evaluation algorithm g the popular Inductive Logic Programming (ILP) em Aleph. We use a neural network to approximate hypothesis-evaluation function. The trained neural ork replaces Aleph’s hypothesis evaluation on actual , scoring potential rules in time independent of the ber of examples. Our approximate evaluator can be used in a heuristic search to help escape local ima. We test the neural network's ability in learning hypothesis-evaluation function on four benchmark ILP ains; the neural network is able to accurately roximate the hypothesis-evaluation function. ntroduction and Background ost data mining techniques assume that the data ts in a form that can be easily converted into a set of d-length feature vectors (where each example is verted into a fixed-size array of real numbers, integers, nominal attributes). For many multi-relational sets, such a conversion – when even possible – is egant and scales poorly. Conversely, Inductive Logic gramming (ILP) [1] natively handles multi-relational . ILP's natural treatment of multi-relational datasets ids the problems associated with converting examples feature vectors. As a further advantage, its rules have full expressive power of first-order logic, making for and human-readable hypotheses. LP systems have been proven quite successful in structing a set of accurate rules, even on datasets with y relations. Such systems have been successfully loyed in a number of varied domains, including molecular biology, engineering design, natural language processing, and software analysis. ILP systems combine background domain knowledge and categorized training data in constructing a set of rules (hypotheses) in first-order logic. Formally, given a training set of positive examples E, negative examples E, and background knowledge B, all as set of clauses in first-order logic, ILP's goal is finding a hypothesis (a set of clauses in first-order logic) h, such that − + ⇒/ ∪ ⇒ ∪ E h B E h B (1) That is, given the background knowledge and the hypothesis, one can deduce all of the positive examples, and none of the negative examples. In real world applications, these constraints are usually relaxed somewhat, allowing h to explain most positive examples and few negative examples. The algorithm underlying most ILP systems is basically the same. It searches for a clause in the subsumption lattice [2], evaluating candidate clauses on the training data. The search begins with an initial candidate clause, and considers hypothesis generation as a local search problem in the subsumption lattice. The starting point for the search and the type of local search depends on the implementation of the ILP system. The subsumption lattice is constructed based on the idea of specificity of clauses. Specificity here refers to implication; a clause C is more specific than a clause S if S ⇒ C. In general, it is undecidable whether or not one clause in first-order logic implies another [3], so ILP systems use the weaker notion of Plotkin's θsubsumption. Subsumption implies implication, but implication does not imply subsumption. Subsumption of candidate clauses puts a partial ordering on all clauses in hypothesis space. With this partial ordering, a lattice of clauses can be built. ILP implementations perform some type of local search over this lattice when considering candidate hypotheses. Most ILP implementations also use a standard greedy covering algorithm. After completing a local search of the subsumption lattice, the best rule evaluated is accepted, and all the positive examples covered (explained) by the rule are removed from the dataset. The process is repeated until every positive example is covered. The major distinction separating various ILP implementations is the strategy used in exploring the subsumption lattice. Algorithms fall into two main categories (with some exceptions): general-to-specific ("top-down") [4] and specific-to-general ("bottom-up") enumeration of the subsumption lattice [5]. Within this framework, a variety of common local search strategies have been employed, including breadth-first search [6], depth-first search, heuristic-guided hill-climbing variants [5,6], uniform random sampling [7], and rapid random restarts [8]. Our work provides a general framework for increasing the speed of any ILP algorithm, regardless of the order candidate clauses are evaluated. One complaint levied against ILP systems is that they scale poorly to large datasets. Srinivasan [7] investigated the performance of ILP algorithms in general, and found that the worst-case running-time depends on both the size of the subsumption lattice and the time required for clause evaluation. The first factor – the search space size – depends on the maximum allowed clause length and the number of terms in an example's saturation. The idea of saturation is used by a number of ILP systems to put a bound on the size of the subsumption lattice. Saturation involves first choosing a positive example from the training set. Using the background knowledge, saturation constructs the most specific, fullyground clause that entails the chosen example. It is constructed by applying all possible substitutions for variables in B with ground terms in B. This clause is called the chosen example's bottom clause, and it serves as the bottom element (⊥) in the subsumption lattice over which ILP searches. That is, all clauses considered by ILP (in the subsumption lattice) subsume (and thus imply) the saturated example. As a simple example, suppose we are given background knowledge (using Prolog notation where ground atoms are denoted with an initial lowercase letter and variables are denoted with an initial uppercase letter):